1. Field of the Invention
The invention relates to a method and a device for processing a digitized image.
2. Description of the Related Art
In the digital transmission of an image, particularly following image compressionxe2x80x94that is to say, the reduction of image. data that are to be transmittedxe2x80x94the channel over which the transmission is accomplished is a bottleneck. The image data are transmitted over this channel from an (image) encoder to an (image) decoder. The bandwidth of this channel is mostly predetermined and constant, that the compression is correspondingly tailored to this bandwidth in the coder. According to the possibilities opened up by the standardized block-based image coding methods, for instance MPEG 4 or H.263 (see International Telecommunication Union, Telecommunications Sector of ITU, Draft ITU-T Recommendation H.263, Videocoding for low bitrate communication, May 2, 1996), the coder can be preset, or respectively, adapted, such that the number of images transmitted per unit of time is guaranteed, in exchange for which a loss of image quality is accepted.
One example is the videotelephone, which displays images that are transmitted over an ISDN B-channel with a transmission rate of 64 Kbit/s and that comprise a corresponding image quality: The images always produce an abrupt sequence having low resolution, and the images are displayed relatively small.
Block-based image coding methods (e.g. in accordance with MPEG or H.263 standards) are known from J. De Lameillieure, R. Schafer MPEG-2-Bildcodierng fxc3xcr das xe2x80x98digitale Fernsehenxe2x80x99, Fernseh-und Kino-Technik.
In image processing, the image quality is specified for a complete image that is to be transmitted. The image quality is modified depending on the available data rate, so that the bandwidth is exploited and the entire image can be transmitted. The disadvantage here is that a loss of quality must be accepted for the entire transmitted image.
It is the object of an invention to create a method and an arrangement for image processing whereby the above described disadvantage is avoided.
The object is achieved by a method for image processing, in which a digitized image is divided into two regions, whereby a first region satisfies a predetermined criterion, and a second region does not satisfy the predetermined criterion; in which the first region is processed with a higher image quality than the second region; in which the image quality is achieved in that the first region is quantized using a first quantization value, and the second region is quantized using a second quantization value, the first quantization value being less than the second quantization value; in which, when the criterion is satisfied, an optimally large contiguous region comprising several blocks that has the predetermined color (skin color) is guaranteed in that one of the following possibilities is realized: when a block does not satisfy the criterion, and this block is largely surrounded by additional blocks that do satisfy the criterion, this block is also coded with the first quantization value if the result of the comparison operation lies slightly over the predetermined threshold value; and when a block does not satisfy the criterion, and this block is situated amid blocks that do satisfy the criterion, then this block is also coded with the first quantization value.
The invention also provides an arrangement for image processing with a processor unit which is laid out such that a digitized image is divided into two regions, whereby a first region satisfies a predetermined criterion, and a second region does not satisfy the predetermined criterion; the first region is processed with a higher image quality than the second region; the image quality is achieved in that the first region is quantized using a first quantization value, and the second region is quantized using a second quantization value, the first quantization value being less than the second quantization value; in which, when the criterion is satisfied, an optimally large contiguous region comprising several blocks which has the predetermined color (skin color) is guaranteed in that one of the following possibilities is realized: when a block does not satisfy the criterion, and this block is largely surrounded by additional blocks that do satisfy the criterion, this block is also coded with the first quantization value if the result of the comparison operation lies slightly over the predetermined threshold value; and when a block does not satisfy the criterion, and this block is situated amid blocks that do satisfy the criterion, then this block is also coded with the first quantization value.
A method for image processing is designed by which a digitized image is divided into two regions: a first region which satisfies a predetermined criterion, and a second region which does not satisfy the predetermined criterion. The first region is then processed with a higher image quality.
An advantage of this is that a better image quality is achieved for the first region, which satisfies the predetermined criterion. The invention thus makes it possible to electronically processxe2x80x94that is, to transmit or compressxe2x80x94the first region as a part of the total image with a higher quality.
A development of the invention consists in dividing the image into several regions, whereby a number of first regions that satisfy the criterion are processed with a higher image quality than a number of second regions that do not satisfy the criterion.
This makes it possible to subdivide numerous regions within an image into several first and several second regions, and thus also to assign a higher image quality to noncontiguous regions of the same type as the first regions in the image.
A possible development of the invention is realized in that the criterion is satisfied when the first region of the image has a predetermined color. This can be a color similar to the color of human skin, for example.
A further development is realized in that the image is processed using a block-based image coding method. Examples of block-based image coding methods are methods that are defined in accordance with the MPEG or H.263 standards.
In a block-based coding method, for a block of the image of a predetermined size the color of the block is determined in the form of second color values, preferably by averaging the image points of this block. A comparison operation of the second color values to the first color values is performed. If a result of the comparison operation is less than a predetermined threshold value, the criterion is satisfied; that is, the block has a color that is at least similar to human skin. Otherwise (that is, the result of the comparison operation is not less than the predetermined threshold value), the criterion is not satisfied for this block.
A further development consists in carrying out the method iteratively for each block of the image.
The predetermined size of the block preferably amounts to 8xc3x978 image points or 16xc3x9716 image points.
The comparison operation can be defined in different ways. Three possibilities are given below (see equations (1) to (3)):
|xyxe2x88x92hy|+|xCrxe2x88x92hCr|xe2x80x94|xCbxe2x88x92hCb| less than Sxe2x80x83xe2x80x83(1),
|xyxe2x88x92hy|2 +|xCrxe2x88x92hCr|2+|xCbxe2x88x92hCb|2 less than Sxe2x80x83xe2x80x83(2),
kixc2x7|D1|+k2xc2x7|D2|+k3xc2x7|D3| less than S, xe2x80x83xe2x80x83(3),
where
xy references a luminance value (=brightness) of the first color values;
xCr references a first chrominance value (=color tone) of the first color values;
xCb references a second chrominance value (=color saturation) of the first color values;
hy references a luminance value (=brightness) of the second-color values;
hCr references a first chrominance value (=color tone) of the second color values;
hCb references a second chrominance value (=color saturation) of the second color values;
S references the predetermined threshold;
k1,k2,k3 reference predeterminable weights;
D1 references a first comparison of the luminance value of the first color values to a luminance value of the second color values;
D2 references a second comparison of a first chrominance value of the first color values to a first chrominance value of the second color values;
D3 references a third comparison of a second chrominance value of the first color values to a second chrominance value of the second color values.
It is particularly advantageous that the three quantities (luminance value, first chrominance value, second chrominance value) for an image point, or respectively, for an image block (correspondingly averaged from the image points of the image block), are compared to the predetermined threshold simultaneously. It is likewise conceivable to compare brightness, color tone, and color saturation for each block of the image, respectively, to a correspondingly predetermined value pertaining to skin color. In this case, one obtains three results of three comparison operations, which can be used to decide whether or not the criterion is satisfied. In other words, one obtains the results of individual comparisons D1, D2 and D3. In the foregoing equations (1) to (3) the three individual comparison values are combined with one another and are collectively compared to the predetermined threshold value.
It is also possible to use a block-based image coding method in accordance with the MPEG 4 standard, whereby this standard makes it possible to make transmission modalities compatible for a defined region (what is known as an image object)xe2x80x94namely, one which preferably satisfies the predetermined criterion.
Another development is realized in that the higher image quality is achieved by specifying a higher image repetition rate for the first region. A selective image repetition rate for the first region (or several first regions in the digitized image, accordingly) that is higher than an image repetition rate for the (usually one) second region makes possible a more frequent updating of the first region, and thus a more fluid representation of movements. If a face is to be displayed, which represents a speaker, for example, the viewer sees the lip movements in continuous moving images, while the background, which does not correspond to the color of skin, is updated far less frequently, and so movements on this background are perceived only abruptly.
It is also possible to improve the image quality for the first region by increasing the topical resolution for this first region. A higher number of image points (pixels) compared to the second region guarantees a clearer rendering (higher resolution) of the first region.
A third possibility for improving the image quality is to influence the quantization values for both the first and second regions. The first region is quantized with a first quantization value; the second region, with a second quantization value. A higher image quality of the first region is guaranteed by the fact that the first quantization value is less than the second quantization value.
In the context of a further development, it is possible in accordance with the MPEG standard to switch over from the first quantization value (low valuexe2x86x92high resolutionxe2x86x92high demand for bandwidth) to the second quantization value (high valuexe2x86x92low resolutionxe2x86x92low demand for bandwidth), (the first quantization value being less than the second quantization value), in that a DQUANT symbol is transmitted with the second quantization value as a parameter; and conversely, it is possible to switch over from the second quantization value to the first quantization value in that the DQUANT symbol is transmitted with the first quantization value as the parameter.
It is also a development of the invention to guarantee an optimally large contiguous region as the first or second region, since this keeps down the number of changeover processes between the quantization values. As described above, each changeover process is preceded by a DQUANT symbol, by which an overhead is created, which is responsible for a loss of bandwidth that could otherwise be used for image information.
When, for example, several blocks of the image which satisfy the criterion are processed with the corresponding quantization value, an individual block situated between these several blocks that exceeds the predetermined threshold value should be quantized according to one of the equations (1) to (3) with a different quantization value than the image blocks that satisfy the criterion; the individual block, for which an extra quantization value would have to be newly declared, is thus also processed with the lower quantization value, like the blocks that satisfy the criterion.
One object is to minimize the changeover processes between quantization values that apply to consecutive blocks of the image. An individual block amid blocks that satisfy the criterion is advantageously also processed with the low quantization value and is thus transmitted with correspondingly high image quality, like the image blocks that surround it.
It is advantageous to establish in a preprocessing step prior to the actual quantizing process which blocks satisfy the criterion, which blocks in the vicinity of the blocks that satisfy the criterion do not satisfy the criterion, and which blocks of the image do not satisfy the criterion and are situated at a definite distance to the blocks that do satisfy the criterion. It is then possible, corresponding to the row-by-row processing of the image with the aid of a block-based coding method, to determine a favorable number and distribution of the changeover processes between the quantization values in that optimally large contiguous regions are acquired as blocks of the image that do or do not satisfy the criterion, respectively.
To this end, the threshold value S for a block can be controlled dependent on a number-of crossings of this threshold value in the neighboring blocks. The threshold value S is preferably lowered when many of the neighboring blocks comprise the predetermined color, and the threshold value S is raised when few of the neighboring blocks comprise the predetermined color.
It is noted that it is also possible to apply a combination of the aforesaid possibilities for improving image quality. For the most part, the desire for better image quality is subject only to physical framework conditions, such as an available maximum bandwidth. This is dependent only on the respective application and does not limit the combination of possibilities for improving the image quality, particularly of regions of a digitized image.
An advantage of the invention consists in specifying the image quality in an image that is to be transmitted such that the quality varies from location to location. It is highly advantageous to transmit portions of an image that are the color of skin with a higher quality than other parts of the image. An example of this is a telecommunication transmission in which the face and hands of the speaker are quantized and transmitted with a lower quantization value and thus in higher quality than the non-skin-colored region of the image. This also opens up the possibility for hearing impaired people to obtain information from the region of high image quality (skin-colored region), for instance by lip-reading or interpreting gestures. The videotelephone thus becomes a useful medium of communication for speech impaired or hearing impaired people, in particular, since hands as well as the face and (lips) are transmitted in higher quality than the rest of the image, and it becomes possible to communicate by means of lip and hand movements.
An arrangement for processing images is also inventively set forth, which arrangement has a processor unit that is configured such that the above described method can be carried out.
An arrangement for encoding a digital image is further set forth, which has means for carrying out a spectral transformation that transforms the digitized image into the spectral region. A means for entropy coding is also provided, which carries out a data reduction. Finally, the arrangement for coding has a buffer, which accepts data of varying rates from the means for entropy coding and forwards them to a channel which preferably has a fixed data rate.
A further development of the invention is realized in that an occupancy level of the buffer adjusts the quantization value to the device for quantization.
In another development, the device for quantization can be adapted in that a full buffer raises the quantization value and thus effectuates an unsharp transmission of the image, and an empty buffer reduces the quantization value, guaranteeing a higher image quality.
Developments of the invention are provided in which the image is divided into several regions, whereby a number of first regions that satisfy the criterion are processed with a higher image quality than a number of second regions that do not satisfy the criterion. Preferably, the criterion is that a region of the image has a color from a predetermined color domain. The color approximately corresponds to the color of human skin.
The present method is applicable in which the image is processed using a block-based image coding method. The block-based image coding method is defined in accordance with an MPEG standard or the H.261 or H.263 standards. The image encoding provides that, for a block of the image, which is a specific size, the color of the block is determined in the form of second color values of the block; a comparison operation of the second color values to the first color values is performed; when a result of the comparison operation is less than a predetermined threshold value, the criterion is satisfied; that is, the block is at least similar to human skin; and, otherwise, the criterion is not satisfied for this block. In one embodiment, for each block of the image, the method is carried out iteratively. The predetermined size of the block is either eight pixels by eight pixels or 16 pixels by 16 pixels.
The comparison operation may be defined by:
|xyxe2x88x92hy|+|xCRxe2x88x92hCR|+|xCbxe2x88x92hCb| less than S
where
xy references a luminance value (=brightness) of the first color values;
xCr references a first chrominance value (=color tone) of the first color values;
xCb references a second chrominance value (=color saturation) of the first color values;
hy references a luminance value (=brightness) of the second color values;
hCr references a first chrominance value (=color tone) of the second color values;
hCb references a second chrominance value (=color saturation) of the second color values; and
S references the predetermined threshold value.
Alternatively, the comparison operation is defined by:
|xyxe2x88x92hy|2+|xCrxe2x88x92hCr|2+|xCbxe2x88x92hCb|2 less than S,
where
xy references a luminance value (=brightness) of the first color values;
xcr references a first chrominance value (=color tone) of the first color values;
xCb references a second chrominance value (=color saturation) of the first color values;
hy references a luminance value (=brightness) of the second color values;
hCr references a first chrominance value (=color tone) of the second color values;
hCb references a second chrominance value (=color saturation) of the second color values; and
S references the predetermined threshold value.
The comparison operation may instead be defined by:
k1xc2x7|D1|+k2xc2x7|D2|+k3xc2x7|D3| less than S,
where
k1,k2,k3 reference predeterminable weights;
D1 references a first comparison of the luminance value of the first color values to a luminance value of the second color values;
D2 references a second comparison of a first chrominance value of the first color values to a first chrominance value of the second color values;
D3 references a third comparison of a second chrominance value of the first color values to a second chrominance value of the second color values,
S references the predetermined threshold value.
According to one development, the block-based image coding method is defined in accordance with the MPEG-4 standard, and the predetermined criterion is satisfied for a region that is definable in accordance with the MPEG-4 standard.
The higher image quality is achieved in that a higher image repetition rate is specified for the first region. The higher image quality is achieved in that a higher topical resolution is specified for the first region such that a higher number of pixels are processed for the first region. The changeover from the first quantization value to the second quantization value is accomplished in a preferred embodiment in that a DQUANT symbol is transferred with the second quantization value as a parameter, and conversely, the changeover from the second quantization value to the first quantization value is accomplished in that the DQUANT symbol is transferred with the first quantization value as the parameter.